In a parallel version of MATLAB® or Simulink® from The MathWorks, Inc. of Natick, Mass., a single computer program may be executed in multiple technical computing environments on one or more computers or processors. Each of the technical computing environments may perform processing of a portion of a set of data to distribute data processing. That is, the same program may run on multiple technical computing environments but process different portions of a distributed data set or each process a data set from multiple data sets.
In the case of parallel processing of a distributed array, one technical computing environment may process a portion of the array, such as a first column, and a second technical computing environment may process another portion of the array, such as a second column. In some cases, a distributed array may be too large and would take up to much memory space to be stored in one technical computing environment. In other cases, the parallel program does not need access to the entire distributed array, but only some portions and only some of the time or under certain conditions. That is, having an entire copy of the distributed array may be unnecessary and inefficient with respect to the operations of the parallel program.
The single program running in parallel in each technical computing environment may perform an operation or calculation on the data of the distributed array that requires the program and/or technical computing environment to obtain a portion of data from another technical computing environment to complete the operation or calculation. For example, the single computer program may perform an operation which requires calculations involving nearby or adjacent data points, such as finite difference methods, convolutions, and local image processing. In these types of operations, a technical computing environment may not have the adjacent or nearby data points, or otherwise have the current values of the nearby data points being processed by another technical computing environment. In order to complete the desired operation, the technical computing environment needs to obtain these other data points included in a portion of the distributed array assigned to or stored in another technical computing environment. The technical computing environment may perform message passing to another technical computing environment to obtain a desired portion of the distributed array stored in the other technical computing environment. The message passing consumes computing and network resources.
Furthermore, the technical computing program may perform the same or similar operations on the same data sets which are distributed among multiple technical computing environments. Each time the technical computing program references a portion of the distributed array stored in another technical computing environment, the technical computing environment may need to exchange messages to obtain a copy of the data. This consumes additional computing and network resources to obtain data previously obtained.